A Comparison of Item and Testlet Selection Procedures

نویسندگان

  • Leslie Keng Pearson
  • Tsung-Han Ho
  • Tzu-An Ann Chen
  • Barbara G. Dodd
چکیده

Testlet response theory (TRT) is a measurement model that can capture local dependency in testlet-based tests. One of the purported advantages of TRT over the more commonly-used polytomous IRT approach to modeling testlet-based tests is that it allows for ad hoc testlet construction in a testlet-based computer adaptive test (CAT). The goal of this study was to investigate the merits of such a CAT design. Specifically, it examined the use of testlet-based CATs that not only chose each testlet adaptively, but they also adaptively selected each item within the testlet, based on the estimated examinee proficiency. This design was termed a CAT with adaptive testlets, and it was compared against a CAT whose within-testlet items were all pre-determined and fixed (termed a CAT with fixed testlets). Real data from a large-scale assessment were calibrated using the 3PL-TRT model and used in this simulation study, which compared these testlet-based CAT designs on their measurement and exposure control properties. The study found that the use of adaptive testlets improved measurement precision while achieving better pool utilization rates. The use of an item-level exposure control procedure within a CAT with adaptive testlets resulted in similar measurement precision, but only a modest gain in pool utilization rates, when compared to a CAT with adaptive testlets and no item-level exposure control. This study represented an initial examination of the properties of CATs with adaptive testlets. As such, suggestions for future research are also provided.

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Polytomous multilevel testlet models for testlet-based assessments with complex sampling designs.

Applications of standard item response theory models assume local independence of items and persons. This paper presents polytomous multilevel testlet models for dual dependence due to item and person clustering in testlet-based assessments with clustered samples. Simulation and survey data were analysed with a multilevel partial credit testlet model. This model was compared with three alternat...

متن کامل

A Multilevel Testlet Model for Dual Local Dependence

The applications of item response theory (IRT) models assume local item independence and that examinees are independent of each other. When a representative sample for psychometric analysis is selected using a cluster sampling method in a testlet-based assessment, both local item dependence and local person dependence are likely to be induced. This study proposed a four-level IRT model to simul...

متن کامل

Minimizing the Testlet Effect: Identifying Critical Testlet Features by Means of Tree-Based Regression

Standardized tests often group items around a common stimulus. Such groupings of items are called testlets. The potential dependency among items within a testlet is generally ignored in practice, even though a basic assumption of item response theory (IRT) is that individual items are independent of one another. A technique called tree-based regression (TBR) was applied to identify key features...

متن کامل

Lsac Research Report Series

The Law School Admission Council (LSAC) is a nonprofit corporation that provides unique, state-of-the-art admission products and services to ease the admission process for law schools and their applicants worldwide. More than 200 law schools in the United States, Canada, and Australia are members of the Council and benefit from LSAC's services. may change without notice at any time. Up-to-date ...

متن کامل

Examining testlet effects on the PIRLS 2006 assessment

We fit 10 PIRLS testlets with standard item response and testlet response theory model. The variances of testlet effects are examined and compared. Item parameters and ability parameters estimated from the two different models are compared. The results indicate that variances of testlet effects range from .168 to .489. Item parameters as well as ability parameter estimated from two approaches c...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2008